Augmented reality

ABSTRACT

In a method and system of training, one or more digital assets, such as two-dimensional and three-dimensional computer-generated objects, are superimposed over a live camera view to generate a simulated training scenario, referred to herein as augmented reality (“AR”) technology. By leveraging AR technology, a live simulation of real-world events, situations, and skills is generated for which an employee, student, customer, or any type of person in need of training, is being trained. A trainee is thus immersed directly into the training material. The physical environment and working conditions are integrated into the sequence of training material, and it does this live, that is, with immediate feedback from a camera&#39;s live screen view. This technique may, by way of examples, also be described as an automated guided tour used to facilitate employee on-boarding and training, or as a guided tour through a warehouse store for its customers.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application No.62/130,391, filed Mar. 9, 2015, which application is hereby incorporatedherein by reference, in its entirety.

TECHNICAL FIELD

The invention relates generally to training and, more particularly, tousing augmented reality to enhance the impact and effectiveness oftraining of persons, such as employees, customers, students, and anyperson or persons in need of training.

BACKGROUND

Retailers and manufacturers are two sectors of the economy that rely onacquiring and retaining a substantial number of unskilled employeesrequired to perform various operations. One of the key challenges inbringing on such employees is to provide them with the knowledge andunderstanding required to perform their new jobs safely and effectively.One of the key strategies used to provide this knowledge is through theuse of training tools such as written training material, classroomgroup-based-training, one-on-one training with a supervisor, and on-linetraining using text, video, photography, and electronic quizzes. Each ofthese techniques suffers from limitations of effectiveness, such as (1)high cost (including direct cost and/or time spent away from one's jobduties), (2) low relevance (e.g., images or videos that are not specificto a respective industry or environment, or that do not, for example,accurately and/or realistically portray dangerous situations), and (3)low impact (e.g., information is not retained for much time afterdelivery to an employee). These limitations also apply to other types oftrainees such as customers shopping in warehouse stores, studentsenrolled in technical courses (i.e., welding, electrical), or any personin need of training.

Therefore, what is needed is a system and method for training people,such as employees, customers, and students, that is cost effective,relevant, and that will have a lasting impact on an employee'sperformance.

SUMMARY

The present invention, accordingly, solves these problems in a uniqueand compelling manner by superimposing one or more digital assets, suchas two-dimensional and three-dimensional computer-generated objects,over a live camera view to generate a simulated training scenario,referred to herein as augmented reality (“AR”). By leveraging ARtechnology, a live simulation is generated of real-world events,situations, and skills for which a person is being trained. In effect, aperson is immersed directly into the training material. The physicalenvironment and working conditions are integrated into the sequence oftraining material, and it does this live, that is, with immediatefeedback from a camera's live screen view. This technique may also bedescribed as an automated guided tour used to facilitate employeeon-boarding and student training.

The foregoing has outlined rather broadly the features and technicaladvantages of the present invention in order that the detaileddescription of the invention that follows may be better understood.Additional features and advantages of the invention will be describedhereinafter which form the subject of the claims of the invention. Itshould be appreciated by those skilled in the art that the conceptionand the specific embodiment disclosed may be readily utilized as a basisfor modifying or designing other structures for carrying out the samepurposes of the present invention. It should also be realized by thoseskilled in the art that such equivalent constructions do not depart fromthe spirit and scope of the invention as set forth in the appendedclaims.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of the present invention, and theadvantages thereof, reference is now made to the following descriptionstaken in conjunction with the accompanying drawings, in which:

FIG. 1 exemplifies a schematic overview of software architecture andfeatures of the present invention;

FIG. 2 exemplifies a training session of the system of FIG. 1, thesession having a number of lessons;

FIG. 3 exemplifies a camera view of a juice storage bin of FIG. 1;

FIG. 4 depicts a live camera view of FIG. 3;

FIG. 5 exemplifies a trigger image of FIGS. 3 and 4;

FIG. 6 exemplifies an augmented camera view of the live camera view ofFIG. 4;

FIG. 7 shows a live camera view exemplified in an additional applicationof the invention;

FIG. 8 exemplifies an augmented camera view of the live camera view ofFIG. 8;

FIG. 9 exemplifies a quiz subsequent to the lesson of FIGS. 3-8;

FIG. 10 depicts a flow chart of steps executable in accordance with theinvention for training; and

FIG. 11 depicts a flow chart of steps executable in accordance with theinvention for displaying a training situation.

DETAILED DESCRIPTION

The following description is presented to enable any person skilled inthe art to make and use the invention, and is provided in the context ofa particular application and its requirements. Various modifications tothe disclosed embodiments will be readily apparent to those skilled inthe art, and the general principles defined herein may be applied toother embodiments and applications without departing from the spirit andscope of the present invention. Thus, the present invention is notintended to be limited to the embodiments shown, but is to be accordedthe widest scope consistent with the principles and features disclosedherein. Additionally, as used herein, the term “substantially” is to beconstrued as a term of approximation.

It is noted that, unless indicated otherwise, functions described hereinmay be performed by a processor such as a microprocessor, a controller,a microcontroller, an application-specific integrated circuit (ASIC), anelectronic data processor, a computer, or the like, in accordance withcode, such as program code, software, integrated circuits, and/or thelike that are coded to perform such functions. Furthermore, it isconsidered that the design, development, and implementation details ofall such code would be apparent to a person having ordinary skill in theart based upon a review of the present description of the invention.Such a person having ordinary skill in the art would be able to make useof commercially-available software tools, components, and libraries tobuild a software application that implements the system being described.

Referring to FIG. 1 of the drawings, the reference numeral 100 generallydesignates a mobile computing device, represented herein as a tablet,configured to embody features of the present invention. The tablet 100includes a central processing unit (also referred to herein as a “CPU”or “processor”) 101 coupled to a memory 102 having an applicationsoftware program 103 executable by processor 101 for training persons,as described in further detail below. A display 108 is coupled via agraphics rendering engine 110 to CPU 101. Display 108 may include adisplay built into tablet 100 or alternative displays, such as anoptical head-mounted display (OHMD, e.g., Google Glass) or the like. Oneor more speakers 112 are preferably coupled via audio hardware 114 toCPU 101.

Tablet 100 includes position sensors 116, such as gyroscopes, which areeffective for generating data indicative of the location and orientationof tablet 100 relative to the target image, or object, 128. Positionsensors 116 are coupled (through CPU 101) to memory 102 for inputtingthe position data to tablet location orientation software module 118which is run by CPU 101 through application software program 103 fordetermining the location and orientation of tablet 100 relative to thetarget image, or object, 128, and saving that information into thememory 102. A camera 120 is mounted on tablet 100 for capturing a cameraview 121, preferably live, of an object 128, exemplified as a juicestorage bin having a trigger image 130 and the object's environment, andfor generating image data indicative of camera view 121 captured by thecamera. Camera 120 is coupled (through CPU 101) to memory 102 forinputting the image data to an image recognition software engine 122which generates an image signal to an image location orientationsoftware module 124, which is run by CPU 101 through applicationsoftware program 103 for determining the position and orientation ofobject 128 of the image and saving that information into the memory 102.By way of example, with location and orientation of the tablet andtarget image determined, if the tablet is three feet away from thetarget image 128, then the 3D object (augmentation) is rendered acertain size. If the tablet is further from the target image, then the3D object is rendered smaller in size, just like a real object. Withrespect to orientation, if the tablet is above the target image, theaugmentation is rendered as if looking down at the target image fromabove. If the tablet is looking at the target image from the side, thenthe augmentation is rendered as if looking at that side of the targetimage. It is understood that FIG. 1 is a schematic drawing and, as such,camera 120 and tablet position sensors 116 are actually coupled tomemory 102 through the CPU 101.

As further shown in FIG. 1, in memory 102, application software program103 is operative with AR sequence definitions 106, which are trainingprograms designed for a specific purpose or job role. By way of example,but not limitation, AR sequence definitions may be designed for “NewEmployee Training at Retail Grocery” or “Poultry Processing EmployeeTraining.” Each AR sequence definition 106 comprises a number ofsequences, also known as lessons, such as “How to properly clean a delislicer” or “How (and why) to clean up a water spill”). Each sequence, orlesson, comprises one or more steps, each of which steps comprises oneor more digital media, or assets, 104.

Digital assets 104 include one or more of 2D and 3D objects, audio clips(e.g., of human voice instructions or procedures, sample sounds ofmachinery or devices, music, and the like), video clips (e.g., ofinstructions, procedures, dramatizations of problems or incidents,corporate messaging, and the like), static images (e.g., of technicaldrawings, depictions or photographs of machinery, equipment,illustrations, photographs of problems of incidents, and the like), textdata (e.g., of instructions, procedures, statistics, and the like),animations (e.g., of instructions, procedures, and the like), hyperlinks(e.g., to documentation, reports, external applications, and the like),and any other types of digital media.

FIG. 2 of the drawings exemplifies an overview of the system of FIG. 1for executing a training session having a number of lessons. The systemis configured for training an employee 126, also referred to herein as atrainee, having tablet 100 configured to instruct the trainee to startthe training application and follow the instructions that appear on thescreen of the tablet. This is one specific example of the application ofthis technology to provide an effective on-boarding program to newemployees of a retail grocery store.

As shown in FIG. 2, the application preferably instructs employee 126 togo to a specific location in the work environment. The location could bea site of a particular machine, piece of equipment, or component of aprocess. The employee walks to that particular location in the workenvironment. The employee is preferably not accompanied by a manager,supervisor, or any other employee. The employee preferably proceedsthrough the lesson in a self-guided manner. As exemplified in FIG. 2,and discussed in further detail below with respect to FIGS. 3-9, theexemplified training session includes a number of lessons, such as howto respond to a water spill (lesson 1), how to handle bacterial growthon a deli slicer 134 (lesson 2), and the like.

Referring to FIGS. 3-5, as employee 126 reaches the required locationfor training, tablet 100 training application software program 103preferably displays on display 108 a live camera view 140 (FIG. 4) andinstructs employee 126 to look for a trigger image 130 (FIG. 5) having aparticular pre-defined trigger image tag 132 that is attached to somepart of the equipment. Image tag 132 is preferably a barcode, a QR code,a customer's store logo, or any kind of uniquely identifiable image thatmay reside on a sticker or label visibly applied to equipment that isthe object of training.

Once the training application software program 103 detects image tag 132in the camera's live view 140, the training application programpreferably generates an AR overlay 142, using 3D renderings 144 and 146selected from digital assets 104, on top of the camera view 121 (FIG. 6)that the trainee views on display 108. The AR overlay 142 preferablycomprises digital assets 104, including images, text, video, 3Drenderings, sound effects, vibration, animated 3D renderings, and/or thelike. By using the AR overlay elements, digital assets 104, the trainingapplication software program 103 provides a training sequence, orlesson, that incorporates elements of the live camera view 140 of thetrainee's environment to teach the subject matter of the lesson. It maybe appreciated that the invention can provide a simulation and impact ofan unsafe, threatening, and/or costly situation, or of a complex,multi-step procedure, without the cost or actual exposure to theemployee of those actual conditions. By way of examples, but notlimitations, such training sequences, or lessons may include:

1. A water spill on the floor next to some type of cooling equipment.This is exemplified in FIGS. 1-5, wherein a water-monster tentacle 146(FIG. 6) is animated reaching up through a puddle 144 to threaten anemployee. In the case of the water spill, the system can render multipletools, as exemplified in FIG. 9, that could be used to clean up thespill, such as a mop, paper towels, squeegee, etc. These tools arepreferably rendered in proportionate scale and proper orientationrelative to actual objects in the physical environment, enhancing therealism of the training exercise;

2. A contaminated piece of food cutting equipment. By way of example, adeli slicer 134, shown as clean in FIGS. 2 and 7, is rendered in FIG. 8with bacteria on the surface of the cutting blade 164 greatly magnifiedto make the bacteria visible to the eye.

3. A theft in progress in a retail environment can be simulated muchmore cost-effectively than a training exercise setup that involves otheremployees or paid actors.

4. A fire in a chemical storage cabinet.

5. An event external to the work environment, such as a dangerousweather event or loss of power.

6. A multi-step procedure to safely disassemble, clean, and reassemble acommercial-quality electric deli meat slicer.

7. Spills of poisonous liquids.

8. Broken glass on a floor.

9. Contaminated food.

10. Confined spaces.

11. Equipment accidents and failures.

12. A simulation, or 3D rendering, of equipment to train on that is notpresent in the trainee's work environment for various reasons, such asthe equipment being only occasionally available or usable, out forrepairs, the equipment being rental equipment, or the like.

By integrating sounds, sights, and conditions of the actual workenvironment into the simulation, the simulation becomes as realistic aspossible, and is therefore much more effective in achieving suspensionof disbelief. It is more convincing to the trainee that they areactually seeing the event happen in front of them and that they areactively participating in it, instead of merely watching it passivelyfrom a distance and at a different time. In the above example of a theftin progress in a retail environment, the trainee sees for him or herselfthe complexity of a theft situation when there are other employees andeven customers in the same environment that could be affected by thesituation at hand. In such a case, the trainee is much more likely to beemotionally invested in the situation, and to be deeply motivated to payattention to the lesson being taught and especially to the remedies andsolutions that the training provides.

Depending on the specific lesson, the AR digital assets 104 can be usedto simulate the tools and techniques required to help remedy thesituation in the lesson being taught.

Operation of the system is exemplified by flow charts 200 and 300,depicted in FIGS. 10 and 11, respectively. Accordingly, in step 202 ofFIG. 10, a trainee is presented with a set of instructions about whatthey should do and what tools they should use to remedy the situation,as discussed in further detail in the following, with respect to FIG.11.

Referring to FIG. 11, in operation, at steps 302 and 304, camera view121 is passed to the image recognition engine 122 which recognizesobject 128 and detects a trigger image 130, determines the triggerimage's location and orientation, which location and orientation isfixed, and passes that information to application software program 103.At substantially the same time, at step 306, the tablet's positionsensors 116 and location and orientation module 118 provide the locationand orientation of the tablet and target image 128, and passes thatinformation to program 103. When both the trigger image's fixed locationand orientation as well as the tablet's location and orientation areknown, at step 308, execution by program 103 of the AR sequencedefinition 106 begins.

Execution of the AR sequence definition, or lesson, 106 comprises aseries of one or more steps in the AR training course, each of whichsteps calls up one or more digital assets 104. Execution of the firststep or next step in the series of steps begins at step 310. As shown atstep 312, each step of the execution generates a composite visual outputcomprising the original camera view 121, with one or more of the abovedigital assets 104 superimposed over the camera view and, at step 314,the 2D and 3D objects are adjusted in location and orientation such thattheir apparent position orientation remains fixed with respect to thephysical objects in the scene. At step 316, the composite view is passedon to the graphics rendering engine 118 of the tablet and is output tothe tablet's visual display 108. At step 318, audio assets 104 are sentto the audio hardware 114 to be played on the tablet's speakers 112 incoordinated sequence with the visual objects. At step 320, adetermination is made whether there are more steps in the lesson. Ifthere are more steps, execution returns to step 308 to thereby guide thetrainee from one step to the next, in exactly the correct order asrequired by the procedure and as built into the application softwareprogram 103 and AR sequence definition 106. If there are no furthersteps for this lesson, execution proceeds to step 204 of FIG. 10.

At step 204, the trainee is given a quiz (e.g., FIG. 9) and prompted toanswer one or more questions that test their understanding of thematerial of the lesson, based on the situation that is being simulated.Such questions may include 3D-rendered objects that simulate optionsavailable to address a problem of a given situation. Questions may eveninclude multiple correct choices, each of which may reduce or eliminatethe problem, and not necessarily in the same way as other correctoptions.

At step 206, if the trainee chooses an incorrect option, executionproceeds to step 208 wherein the AR digital assets 104 are preferablyused to simulate the effects of the incorrect choice. The effects couldbe that the problem does not get remedied, or the problem gets evenworse, or a new and possibly more dangerous problem is created. 3Danimated renderings and audio effects may be generated to simulate theresult of an incorrect choice in a multiple-choice question, andillustrate to the trainee how an incorrect choice either has no effecton a problem, or causes the problem to get worse, or causes another,potentially worse problem to arise.

At step 210, the application returns to step 202 and enters aremediation loop to re-present the material, possibly in an alternativeformat, order, or style, preferably adapting to the user's personallearning style and preferences, to reinforce the required lesson.

If at step 206, a trainee correctly answers the one or more questions,that lesson is counted as complete and, optionally, 3D animatedrenderings and audio effects are displayed to simulate the result of oneor more correct responses and illustrate to the trainee how one or morecorrect responses cause the problem to be solved or ill effects to beminimized. A determination is then made at step 212 whether there aremore lessons for the topic at hand. If it is determined that there aremore lessons, execution proceeds to step 214, wherein the applicationinstructs the trainee to move to another location in the workenvironment where the next lesson will be displayed. For example, withreference to FIG. 2, a trainee may move from lesson 1 to lesson 2.Execution then returns to step 202. The trainee thus moves through aseries of lessons as described above that comprise the set of lessonsrequired for a specific topic.

If, at step 212, it is determined that there are no more lessons tocover, then at step 216, the application software program 103 will storeas many lessons as necessary for a specific topic. After visiting all ofthe image tags and completing the sequence of lessons, the traineecompletes the training session.

At step 218, the results of the training are uploaded to a cloud-basedservice and stored. The results are then analyzed for particularweaknesses in the trainee's results. A report is then generated for thetrainee's supervisor. The report will direct a supervisor to provide, ifnecessary, further remedial training by means of a talk for the specificareas of weakness. No remedial actions are generated or suggested fortopics for which the trainee exhibited satisfactory results.

Optionally, upon completion of the entire sequence of steps, the finalresult of the procedure may be simulated as a set of 3D renderings andaudio effects and the like superimposed on the live view, the audioeffects being played through the speakers incorporated into the mobilecomputing device, thereby providing positive feedback to the trainee.

It is understood that the present invention may take many forms andembodiments. Accordingly, several variations may be made in theforegoing without departing from the spirit or the scope of theinvention. For example, the training system and method may be configuredto adapt to incorrect choices and an employee's learning style andpreferences. In another example, instructions and orientation may beauditory, that is, verbal. In still another example, the system andmethod may be used to screen or weed out high-risk candidates (e.g.,based on a report of results in step 218) before they are hired. In yetanother example, the application may be completely self-contained,without any capability to upload data to a cloud-based or server-basedcentral application, but instead contains within itself additionalfunctions to generate reports and summaries for supervisor or instructorreview.

By use of the present invention, an automated training system provideson-boarding and continuous learning, using abuilt-as-a-mobile-application which a trainee can use and followcompletely on their own with little or no supervision or guidance. Therelevance, effectiveness, and impact of the teaching material isenhanced, while training costs are reduced.

Further, training lessons are made more realistic by incorporatingelements of the actual physical environment into the lesson, whereinelements include at least one of objects or equipment, employees,customers, lighting levels, noise levels, smell, vibration, andtemperature, and the like.

Still further, the 3D renderings and audio effects generate anexaggerated perspective of a situation that makes the lesson morebelievable as an actual live event, instead of a recording or simulationof an event, and thereby make the training more memorable.

Still further, multiple mobile devices may be configured to enable asingle supervisor, instructor, or customer representative to providemultiple training sessions in parallel.

Still further, a user could be prompted with 3D animated renderings tochoose among a series of options, including at least one of amultiple-choice question that represents real-world strategies toresolve a situation presented in a lesson. By way of example, a waterspill scenario could show three options: a mop, a broom, and papertowels. Instead of rendering these options as merely text or images,they could be rendered as 3D objects. When selected, the objects wouldbe animated to show the result of the use of that tool. Selecting thebroom would show the broom moving the water back and forth, but noprogress made in collecting the water. Selecting the mop would show themop circling the spill and absorbing the liquid. Having thus describedthe present invention by reference to certain of its preferredembodiments, it is noted that the embodiments disclosed are illustrativerather than limiting in nature and that a wide range of variations,modifications, changes, and substitutions are contemplated in theforegoing disclosure and, in some instances, some features of thepresent invention may be employed without a corresponding use of theother features. Many such variations and modifications may be consideredobvious and desirable by those skilled in the art based upon a review ofthe foregoing description of preferred embodiments. Accordingly, it isappropriate that the appended claims be construed broadly and in amanner consistent with the scope of the invention.

The invention claimed is:
 1. A training system comprising: a mobilecomputing device (“MCD”) having a processor and a memory configured forstoring an application software program executable by the processor; acamera mounted on the MCD and configured for capturing an image of anobject having a trigger image having a fixed location and orientation;position sensors mounted on the MCD for generating data indicative of alocation and orientation of the MCD relative to the object, and forstorage of same in memory; a display operative by the processor;augmented reality (“AR”) sequence definitions stored in the memory andaccessible by the program for defining steps of a lesson; and digitalassets accessible by the AR sequence definitions; wherein the program isexecutable by the processor for detecting the trigger image anddetermining the fixed location and orientation of the trigger image and,with reference to an AR sequence definition and data input from thecamera and MCD location and orientation data from the position sensors,for overlaying an image from the camera with the digital assets tothereby generate an AR image, for adjusting the digital assets of the ARimage in location and orientation relative to the object, and fordisplaying an adjusted image on the display.
 2. The system of claim 1wherein the camera captures a live view of a trainee's actualenvironment.
 3. The system of claim 1 wherein the camera captures a liveview of a trainee's actual environment, incorporating elements of theactual environment into the view, wherein elements include at least oneof objects, equipment, employees, customers, lighting levels, noiselevels, and vibration.
 4. The system of claim 1, wherein digital assetsincludes at least one of 2D and 3D objects, audio clips, video clips,static images, text data, animations, and hyperlinks.
 5. The system ofclaim 1 further comprising multiple mobile devices configured to enablea single supervisor, instructor, or customer representative to providemultiple training sessions in parallel.
 6. The system of claim 1 whereinthe trigger image comprises a trigger image tag having at least one of abarcode and a QR code.
 7. A method of training using a mobile computingdevice (“MCD”), the method comprising steps, executed by the MCD, of:capturing with the MCD an original image of an object and itsenvironment, the object including a trigger image; recognizing theobject and trigger image, wherein the trigger image has a fixed locationand orientation; determining the fixed location and orientation of thetrigger image; detecting a location and orientation of the MCD; callingup one or more digital assets; generating an augmented reality (“AR”)image that simulates a problem of a given situation, the AR image beinggenerated by superimposing the one or more digital assets over theoriginal image; adjusting the one or more digital assets of the AR imagewith reference to the location and orientation of the MCD such that theone or more digital assets' location and orientation appears fixed withrespect to the object; and displaying the AR image to a trainee.
 8. Themethod of claim 7, wherein the original image includes a live view of atrainee's actual environment, incorporating elements of the actualenvironment into the view, wherein elements include at least one ofobjects or equipment, other employees, customers, lighting levels, noiselevels, and vibration.
 9. The method of claim 7, wherein digital assetsincludes at least one of 2D and 3D objects, audio clips, video clips,static images, text data, animations, and hyperlinks.
 10. The method ofclaim 7, further comprising a step of prompting the trainee to answerquestions that test the trainee's understanding of material based on thesituation.
 11. The method of claim 7, further comprising a step ofprompting the trainee with 3D animated renderings to choose among aseries of options, including at least one of a multiple-choice questionthat represents real-world strategies to resolve a situation presentedin a lesson.
 12. The method of claim 7, further comprising a step ofprompting the trainee to answer questions that test the trainee'sunderstanding of material based on the situation, wherein the questionsinclude 3D-rendered objects that simulate options available to addressthe problem of the given situation.
 13. The method of claim 7, furthercomprising a step of prompting the trainee to answer questions that testthe trainee's understanding of material based on the situation, whereincorrect responses to questions include 3D animated renderings and audioeffects to simulate the result of one or more correct responses in amultiple-choice question, and illustrating to the trainee how one ormore correct responses cause the problem to be solved or ill effects tobe minimized.
 14. The method of claim 7, further comprising a step ofprompting the trainee to answer questions that test the trainee'sunderstanding of material based on the situation, wherein incorrectresponses to questions include 3D animated renderings and audio effectsto simulate the result of an incorrect choice in a multiple-choicequestion, and illustrating to the trainee how an incorrect choice eitherhas no effect on a problem, or causes the problem to get worse than whenit was originally simulated, or causes another problem to arise that ispotentially worse than the problem originally simulated.
 15. The methodof claim 7, further comprising a step of prompting the trainee to answerquestions that test the trainee's understanding of material based on thesituation, wherein, when the trainee chooses an incorrect option, theapplication flows into a remediation loop, which remediation loopdisplays the relevant training material to the trainee once again,optionally in a different order, format, or style, adapting to thetrainee's personal learning style and preferences.
 16. The method ofclaim 7, further comprising a step of prompting the trainee to answerquestions that test the trainee's understanding of material based on thesituation, wherein, when the trainee chooses a correct option, theapplication simulates the effect of eliminating the problem or reducingthe problem to a manageable level, thus providing feedback to thetrainee as a reward for having made the correct choice.
 17. The methodof claim 7, further comprising a step of prompting the trainee to answerquestions that test the trainee's understanding of material based on thesituation, wherein the trainee has available multiple correct choices,one or more of which may be selected, each of which choices can reduceor eliminate the problem, and not necessarily in the same way as othercorrect options.
 18. A training method comprising steps of: displayingon a mobile computing device (“MCD”) a live view of an object andsurrounding environment; detecting a trigger image having a fixedlocation and orientation and comprising at least one of a barcode and aQR code; detecting the fixed location and orientation of the triggerimage; superimposing 3D animated renderings and audio effects over thelive view to thereby generate a superimposed view; adjustingautomatically one or more digital assets of the superimposed view withreference to location and orientation of the object such that thelocation and orientation of the one or more digital assets appears fixedwith respect to the object; and prompting, with the superimposed view, atrainee to follow a pre-defined sequence of steps required to perform aparticular procedure as required for a training lesson.
 19. The trainingmethod of claim 18, further comprising steps of: reading by the traineeof a description of at least one particular step; and interacting withobjects displayed on the MCD to trigger an animation or visualization ofthe at least one particular step.
 20. The training method of claim 18,further comprising the step of guiding the trainee from one step to thenext, in exactly the correct order as required by a procedure and asbuilt into an application.
 21. The training method of claim 18, furthercomprising the step wherein, upon completion of an entire sequence ofsteps, simulating a final result of a procedure as a set of 3Drenderings and audio effects superimposed on the live view, the audioeffects being played through speakers incorporated into the MCD, therebyproviding feedback to the trainee.